Mission Specific Embedded Training Using Mixed Reality

High risk special operations often require that the team is trained in advance in an environment similar to the target area. It would be ideal if a replica could be built and the team could be trained in the operational environment. However, it is usually difficult to do so in a timely manner, especially when intelligence constantly changes. Traditionally, a scaled-down model of the target area is built instead. Mixed reality based embedded training with the ability to rapidly build and improve the model is a useful tool that can meet these challenges. Several research groups have considered augmented reality (AR) or mixed reality as a training tool for military operations in urban terrain. Our group has developed the Battlefield Augmented Reality System (BARSTM), which can be used for a variety of applications, such as situation awareness as well as embedded training. We have since developed a new version of the system that makes use of the state-of-the-art techniques to build missionspecific training systems in a timely manner. While there is still debate on how effective a current AR situation awareness system is in an actual combat situation, we have seen increased interest in AR training among Marines, Air Force special forces, and others. In this paper, a mission specific embedded military training system using mixed reality is presented. The methods for building environmental models and embedding synthetic characters are described. A set of tools has been assembled to build mission specific models. These models together with a modular physical setting can be integrated into the training system which provides a mixture of virtual objects and physical objects. This mixed environment provides a more realistic training than a pure virtual environment, and more flexible training than pure physical settings.